Embodiments of the invention are directed, in general, to imaging systems and, more specifically, to mixed signal 16 bit floating point processors and methods of capturing wide dynamic range images.
An image system converts an optical image focused on an image sensor into electrical signals. Imaging systems have already been employed in a variety of applications, such as medical devices and satellites, as well as microscope and telescope instrumentation. More recently, imaging systems have been utilized in a variety of additional applications such as digital cameras, computer scanners, digital cellular telephones, and digital camcorders. A majority of these applications have utilized charge-coupled-devices (CCDs) as the underlying image sensors. However, CCD-based image sensors are limited or impractical for use in many consumer applications.
For example, CCDs typically are manufactured using four to six inch wafer fabrication lines, whereas many current fabrication processes employ twelve to fourteen inch lines. CCDs thus cannot be fabricated employing conventional Complimentary-Symmetry Metal-Oxide Semiconductor (CMOS) fabrication processes. The mismatch in processing technology for CCDs also precludes the integration of additional on-chip functionality beyond light sensing and charge to voltage conversion for CCDs. As a result, CCDs usually employ independent support chips to perform, for example, CCD control, ND conversion, and signal processing, such as implemented with CMOS integrated circuits (ICs).
The operation of CCD image sensors also requires multiple high supply voltages (e.g., 5 V to 12 V) resulting in higher power consumption than CMOS imagers. As a result, the costs for the CCD image sensor and the system employing the sensor remain high. Additionally, since the current to charge the CCDs is high, CCDs are not well suited for portable devices.
CMOS image sensors have offered several improvements in functionality, power and cost in many applications (e.g., digital video, digital cameras, and digital cell phones). A CMOS type image sensor includes a photodiode or phototransistor employed as a light detecting element. These sensors often use active pixels, and hence their alternate name Active Pixel Sensors (APS). In APS image sensors and image sensor arrays, each pixel contains an amplifier that converts the collected charge packet to a voltage. The output of the light detecting element is an analog signal whose magnitude is approximately proportional to the amount of light received by the elements. The magnitude of the analog signal can be measured for each photodiode representing a pixel, and then stored to provide an entire stored image. CMOS image sensors utilize less power, have lower fabrications costs, and offer high system integration as compared with image sensors fabricated with CCD processes. Additionally, CMOS image sensors have the advantage that they can be manufactured using similar processes employed to those commonly used to manufacture logic transistors, such that the necessary CMOS image sensor support functions can be fabricated on the same chip.
The potential to achieve wide dynamic range imaging with CMOS image sensors has also attracted attention in the field of electronic imaging, attention that was previously focused on CCDs. Dynamic range is the ratio between the brightest and darkest recordable parts of an image or scene. Several techniques have been utilized in an effort to improve the dynamic range of conventional CMOS image sensors that implement voltage domain sampling. Some of these methods include logarithmic response CMOS image sensors, multiple frame capture techniques, and floating-point pixel-level ADC image sensors. Logarithmic response CMOS image sensors incorporate logarithmic compression at the photodiode level to achieve wide dynamic range. The logarithmic response technique suffers from the problem of fixed pattern noise due to device-to-device mismatches, as well as poor sensitivity and local contrast. Multiple frame capture techniques include the implementation of a lateral overflow gate to increase pixel dynamic range. This technique suffers from mismatch in the lateral overflow transistor gate-drain overlap capacitances. It also requires the capturing and storage of multiple frames in conjunction with complex image reconstruction and processing. Furthermore, its logarithmic compression curve strongly reduces image contrast. Floating-point pixel-level ADC image sensors require large memory to store the data, and also require a complex image reconstruction process.
Along with integration and low power hardware, image processing is continually demanding higher performance of the post-image-acquisition hardware to better process images. As image sensors become more capable with wider dynamic ranges that better match the capabilities of the human eye, most of the available performance of signal processing systems will be required to transform the image into a useable form. With the addition of 3D image sensors, the performance demand is even more significantly increased.
As robotic devices and systems continue to expand in their areas of application, often the system-level performance requirement is to see even better than a human can. To do this, better solutions need to be found to process the images in real time at significantly lower power dissipation. Other areas beyond robotics will have the same demand on the signal and image processing system.
Miniature cameras have also been developed for a wide range of applications, including surveillance, automated inspection, inspection in harsh environments, and certain biomedical applications. For example, an intraocular camera for retinal prostheses is being develop to restore sight to the blind. United States Patent Application 200810086206A1 discloses such an Intraocular Camera for Retinal Prostheses, and is herein incorporated by reference. In all of these applications, the minimization of signal and image processing complexity, along with its associated power dissipation, is of critical importance.
The eye is very different from the present systems used to capture pictures and images. One of the distinct differences is in the dynamic range and the representational precision of the two systems. Typical electrical image capturing systems (i.e., image sensors) have 8 to 10 bits of dynamic range with a similar level of representational precision at a given level of exposure. A mechanical iris is often depended on to handle the bulk of the dynamic range of the real world.
On the other hand, the eye has a dynamic range of about eight orders of magnitude (24 to 26 bits) and a representational precision of about 6 to 8 bits. The real world spans an even greater dynamic range than the eye is capable of, perhaps two to three orders of magnitude (6 to 10 bits) greater. One might suggest that the real world has significantly more information available than our electrical capture systems have the capability of detecting.
Therefore, a need exists for a signal and image processor to capture and store this wide dynamic range (WDR) signal in a standard format, as well as to perform the associated signal and image processing operations efficiently.